Information processing device, information processing method, and information processing program

ABSTRACT

An information processing device includes an acquisition unit, a setting unit, a determination unit, and a control unit. The acquisition unit acquires an operation angle that is an angle formed by a first direction in a predetermined space pointed by a user and a second direction in the predetermined space pointed by the user. The setting unit sets, as a reference angle, the operation angle acquired at a time point when an instruction to start moving a virtual object on a line extending in the first direction is detected. The determination unit determines whether or not the operation angle acquired in response to a change in the second direction is equal to or more than the reference angle. The control unit controls a display unit to move the virtual object in a depth direction on the line in the first direction and display the virtual object while maintaining a distance between an intersection where the first direction and the second direction intersect and the virtual object on the basis of a determination result of the determination unit. Consequently, the position of a distant virtual object can be finely adjusted at a hand position where the operation is easy.

FIELD

The present disclosure relates to an information processing device, aninformation processing method, and an information processing program.

BACKGROUND

For example, in a device for operating a virtual object in athree-dimensional space, improvement regarding localization of a line ofsight is desired in pointing and object operation by a line of sight dueto a human visual adjustment mechanism.

Accordingly, there is known an information processing device thatcontrols a display device so as to display a stereoscopic object that isarranged along a predetermined direction in a visual field of a user andindicates a distance in the predetermined direction, and achievespointing and object operation by the user's line of sight. Consequently,improvement regarding localization of the line of sight is achieved.

CITATION LIST Patent Literature

Patent Literature 1: JP 11-331992 A

Patent Literature 2: JP 2002-44797 A

SUMMARY Technical Problem

In a conventional information processing device, a stereoscopic objectindicating a distance is displayed, and a virtual object can be arrangedwhile visually recognizing a position where the virtual object isdesired to be arranged. However, in the conventional informationprocessing device, in a case where the position where the virtual objectis to be arranged is distant, if the viewing angle changes even a littlewith respect to the position where the virtual object is desired to bearranged, the arrangement position greatly deviates.

Therefore, the present disclosure proposes an information processingdevice or the like capable of finely adjusting the position of a virtualobject located far in a predetermined space.

Solution to Problem

To solve the problems described above, an information processing deviceaccording to an embodiment of the present disclosure includes anacquisition unit, a setting unit, a determination unit, and a controlunit. The acquisition unit acquires an operation angle that is an angleformed by a first direction in a predetermined space pointed by a userand a second direction in the predetermined space pointed by the user.The setting unit sets, as a reference angle, the operation angleacquired at a time point when an instruction to start moving a virtualobject on a line extending in the first direction is detected. Thedetermination unit determines whether or not the operation angleacquired in response to a change in the second direction is equal to ormore than the reference angle. The control unit controls a display unitto move the virtual object in a depth direction on the line in the firstdirection and display the virtual object while maintaining a distancebetween an intersection where the first direction and the seconddirection intersect and the virtual object on the basis of adetermination result of the determination unit.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is views illustrating an example of an information processingsystem according to an embodiment of the present disclosure.

FIG. 2 is a view illustrating an example of a usage form of theinformation processing system.

FIG. 3 is a diagram illustrating an example of the informationprocessing system.

FIG. 4 is a diagram illustrating an example of a functionalconfiguration of an information processing device.

FIG. 5 is a diagram illustrating an example of a direction informationstorage unit.

FIG. 6 is a diagram illustrating an example of an object informationstorage unit.

FIG. 7 is a diagram illustrating an example of an intersectioninformation storage unit.

FIG. 8 is a view illustrating an example of an origin setting process.

FIG. 9 is a view illustrating an example of a reference plane settingprocess.

FIG. 10 is a view illustrating an example of an operation plane settingprocess.

FIG. 11 is a view illustrating an example of an intersection settingprocess.

FIG. 12 is a view illustrating an example of a relationship between amain line and an operation line.

FIG. 13 is a view illustrating an example of a gripping process (withoutan intersection) of a virtual object.

FIG. 14 is views illustrating an example of the gripping process (withan intersection) of the virtual object.

FIG. 15 is a view illustrating an example of a moving process of thevirtual object (distance D1 is less than a threshold).

FIG. 16 is views illustrating an example of the moving process of thevirtual object (distance D1 is equal to or more than the threshold).

FIG. 17 is a flowchart illustrating an example of processing operationof the information processing device related to the moving process.

FIG. 18 is a flowchart illustrating an example of the processingoperation of the information processing device related to the movingprocess.

FIG. 19 is a flowchart illustrating an example of the processingoperation of the information processing device related to the movingprocess.

FIG. 20 is a view illustrating an example of a movement position of thevirtual object at a time of forward movement of the operation line.

FIG. 21 is a view illustrating an example of a movement position of thevirtual object at a time of leftward movement of the operation line.

FIG. 22 is a diagram illustrating an example of the informationprocessing device.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments of the present disclosure will be described indetail with reference to the drawings. Note that in each of thefollowing embodiments, the same parts are denoted by the same referencenumerals, and redundant description will be omitted.

Furthermore, the present disclosure will be described according to thefollowing order of items.

1. Introduction

1-1. Outline of Information Processing System

2. Configuration of Information Processing System of Embodiment

2-1. Configuration of Display Device

2-2. Configuration of Controller

2-3. Configuration of Information Processing Device

2-4. Functional Configuration of Control Unit

3. Operation of Information Processing System

3-1. Intersection Generating Process

3-2. Gripping Process of Virtual Object

3-3. Moving Process of Virtual Object

3-4. Releasing Process of Virtual Object

4. Effects of Embodiment

5. Modification Example

5-1. Other Releasing Processes of Virtual Object

5-2. Other Gripping Processes of Virtual Object

5-3. Other Moving Processes of Virtual Object

5-4. Other Instruction Components

5-5. Other Display Forms of Virtual Object

5-6. Other Master-Slave Relationships

5-7. Other Display Forms of Lines

5-8. Other Geometric Targets

6. Hardware Configuration

7. Conclusion

1. Introduction

<1-1. Outline of Information Processing System>

In a conventional information processing device, a stereoscopic objectindicating a distance is displayed, and a virtual object can be arrangedwhile visually recognizing a position where the virtual object isdesired to be arranged. However, in the conventional informationprocessing device, in a case where the position where the virtual objectis to be arranged is distant, if the viewing angle changes even a littlewith respect to the position where the virtual object is desired to bearranged, the arrangement position greatly deviates.

Therefore, in the conventional information processing device, in a casewhere a stereoscopic object is displayed, there are many restrictions onan instruction of a target such as a position or a virtual object by auser, and it is difficult to perform the instruction by the user ordisplay according to the instruction. Thus, it is desired to enableflexible display according to the user's instruction.

Accordingly, the present applicant proposes an information processingdevice that controls a display device so as to display a mark for avirtual object at an instruction position that is a position determinedon the basis of a plurality of directions pointed by the user.Consequently, the flexible display according to the user's instructionis enabled.

FIG. 1 is a view illustrating an example of information processingaccording to an embodiment of the present disclosure. The informationprocessing according to the embodiment of the present disclosure isimplemented by an information processing device 30 illustrated in FIG. 3. An information processing system 1 illustrated in FIG. 1 includes adisplay device 10, a first controller 20A(20), a second controller20B(20), and an information processing device 30 that controls displayon the display device 10. The display device 10 is, for example, a headmounted display (HMD) mounted on the head of a user X in a real spaceRS. The display device 10 displays an image IM according to the displaycontrol of the information processing device 30 on a display unit 15located in front of the eyes of the user X.

Note that the display device 10 may be a head mounted display such as anon-transmissive HMD, a transmissive HMD, or the like as long as it iscapable of implementing processing to be described later. Furthermore,the display device 10 is not limited to the head mounted display, andmay be any device as long as it is capable of implementing informationprocessing to be described later, and for example, may be variousdevices such as an aerial projection display. Details of theconfiguration of the display device 10 will be described later.

Furthermore, FIG. 1 illustrates a case where the user X carries thefirst controller 20A in the right hand and carries the second controller20B in the left hand. Hereinafter, in a case where the first controller20A and the second controller 20B are not distinguished, they may bedescribed as a controller 20. The controller 20 is a device used by theuser X to point at a direction. The user X points at a desired directionby arranging a hand carrying the controller 20 at a desired position orin a desired orientation. For example, the controller 20 is used toinstruct a position in a space of augmented reality (AR), virtualreality (VR), mixed reality (MR), or the like displayed by the displaydevice 10, or to point at an object (also referred to as a “virtualobject”) in the space.

FIG. 1 illustrates a case where a line is used as an element (target)for allowing the user X to visually recognize the direction pointed bythe user X. For example, a line along the direction pointed by the userX allows the user X to visually recognize the direction pointed by theuser X. In FIG. 1 , a line extending in the direction pointed by theuser X with the controller 20 is displayed on the display device 10,thereby allowing the user X to visually recognize the direction pointedby the user X with the controller 20. FIG. 1 illustrates a case where aline extending from the controller 20 is a virtual beam displayed by thedisplay device 10. For example, the display device 10 displays a lineextending from the controller 20 along an axis passing through apredetermined position (for example, origin 21A1, origin 21B1, and thelike in FIG. 8 ) of the controller 20.

Note that the line extending from the controller 20 is not limited tothe virtual beam displayed by the display device 10, and may be a beam(laser beam) actually emitted by the controller 20. In a case where thecontroller 20 actually emits a laser beam (line), the controller 20emits the laser beam along a predetermined optical axis. Furthermore,the element for allowing the user X to visually recognize the directionpointed by the user X is not limited to a line, and may be a plane orthe like. Hereinafter, a line, a plane, or the like used for allowingthe user X to visually recognize the direction indicated by the user Xmay be collectively described as a geometric target. Furthermore, adirection instruction by the user X is not necessarily performed by thedevice such as the controller 20 but may be performed by the body of theuser X, or the like, and the device, the body of the user X, or the likeused by the user to point to the direction may be collectively describedas an instruction component. The instruction component may be anycomponent as long as it is used by the user X to point to a direction.

The image IM in FIG. 1 is an image displayed in front of the eyes of theuser X by the display of the display device 10 worn by the user X. Inthe image IM in FIG. 1 , a line LN1 corresponding to the directionpointed by the first controller 20A and a line LN2 corresponding to thedirection pointed by the second controller 20B are displayed.

The user X changes a desired position and direction of the hand carryingthe controller 20 while confirming the position and orientation of theline LN1 and the position and orientation of the line LN2, therebydesignating a position determined on the basis of the line LN1 and theline LN2 (hereinafter, also referred to as an “instruction position”).In the example of FIG. 1 , the user X adjusts the orientation andposition of the controller 20 so that the vicinity of a center portionof an image IM1 (the vicinity of the center portion in front of his/hereyes) becomes the instruction position. Hereinafter, adjusting theorientation and position of the controller 20 may be simply described as“adjustment of the controller 20”.

In the example of FIG. 1 , the information processing device 30 controlsthe display device 10 to display a mark MK1 indicating a virtual objectα at the instruction position determined by the user X on the basis ofthe two directions indicated by the controller 20. Here, the informationprocessing device 30 displays the mark MK1 on the basis of thepositional relationship between the two lines LN1 and LN2.

As described above, in the information processing device 30, a desiredposition can be easily instructed by indicating the position by twolines. The information processing device 30 enables flexible displayaccording to an instruction of the user X by controlling the displaydevice 10 to display the mark MK1 at an intersection P determined on thebasis of the lines LN1 and LN2 corresponding respectively to twodirections indicated by the user X. Furthermore, the informationprocessing device 30 switches the display of the intersection Paccording to the distance between the two lines, thereby enablingflexible display corresponding to the instruction of the user X. Theinformation processing system 1 enables the user X to freely designate athree-dimensional position in a three-dimensional space withoutrestriction.

Since the position can be determined while dynamically changing theaxis, the user X can quickly designate the position anywhere in thethree-dimensional space. Furthermore, the user X can create theintersection P by bringing the two lines close to each other in a casewhere he or she wants to designate the position, and can end theintersection display by separating the lines or changing the orientationof the lines in a case where he or she wants to stop designating, sothat the intention of the user X is intuitively reflected.

For example, the user X arranges the virtual object a in thepredetermined space by lines pointing from the respective controllers 20to the predetermined space using the two controllers 20, creates theintersection P by crossing the two lines, and arranges the virtualobject α at the position of the intersection P. That is, when two linesare used, any three-dimensional position in the predetermined space canbe designated.

In the information processing device 30, in a case of pointing in thevicinity of the operation position of the controller 20 of the user X byusing the two controllers 20 that perform pointing in conjunction withmovement of the hand of the user X, the distance change amount of theintersection P with respect to the change in operation angle between thetwo main lines and the operation line can be controlled.

In the present invention, for example, a scene is assumed in which theuser X such as a content creator finely adjusts the three-dimensionalposition of the virtual object a located far in the predetermined space.FIG. 2 is a view illustrating an example of a usage form of theinformation processing system 1.

As illustrated in FIG. 2 , the distance change amount of theintersection P with respect to the operation angle when pointing at thedistant virtual object α is relatively larger than that in the vicinity.The control of the movement distance of the virtual object α based onthe operation angle according to the distance from the operationposition of the user X is a technical problem. That is, even at the sameoperation angle, the amount of movement of the position of the virtualobject α is larger in the distance than in the vicinity, and thus fineadjustment of the position of the distant virtual object α is difficult.

Moreover, in a case where it is attempted to designate a distantposition, the position greatly changes due to slight movement or shakeof the hand, and thus in a case where it is attempted to finely adjustthe position of a distant intersection or the position of the distantvirtual object α, it is difficult to adjust the position.

Therefore, in the present invention, as illustrated in FIG. 2 , there isa demand for the information processing device 30 capable of finelyadjusting the position of the distant intersection P and the position ofthe virtual object α in the predetermined space by using two controllers20, for example, finely adjusting the position of planting (virtualobject α) 50 m ahead so as to move 10 cm toward a near side.

Accordingly, the information processing device 30 includes anacquisition unit, a setting unit, a determination unit, and a controlunit. The acquisition unit acquires an operation angle that is an angleformed by a first direction in a predetermined space pointed by a userand a second direction in the predetermined space pointed by the user.The setting unit sets, as a reference angle, the operation angleacquired at a time point when an instruction to start moving a virtualobject on a line extending in the first direction is detected. Thedetermination unit determines whether or not the operation angleacquired by the acquisition unit in response to a change in the seconddirection is equal to or more than the reference angle. The control unitcontrols the display device 10 to move the virtual object in a depthdirection on the line in the first direction and display the virtualobject while maintaining a distance between an intersection where thefirst direction and the second direction intersect and the virtualobject on the basis of a determination result of the determination unit.

The information processing device 30 determines whether or not theoperation angle acquired in response to the change in the seconddirection is equal to or more than the reference angle, and controls thedisplay device 10 to move the virtual object in the depth direction onthe line in the first direction and display the virtual object whilemaintaining the distance between the intersection where the firstdirection and the second direction intersect and the virtual object onthe basis of the determination result. Consequently, the virtual objectis displayed movably in the depth direction on the line in the firstdirection while maintaining the distance between the intersection andthe virtual object, and thus fine adjustment of the distant virtualobject becomes easy.

2. Configuration of Information Processing System of Embodiment

FIG. 3 is a diagram illustrating an example of the informationprocessing system 1. The information processing system 1 illustrated inFIG. 3 includes the display device 10, the first controller 20A(20), thesecond controller 20B(20), and the information processing device 30. InFIG. 3 , two controllers 20 of the first controller 20A and the secondcontroller 20B are illustrated as an example of the directioninstruction components, but the information processing system 1 mayinclude more than three direction instruction components.

The information processing system 1 is, for example, a system in whichinformation processing related to augmented reality (AR), virtualreality (VR), or mixed reality (MR) is executed. For example, theinformation processing system 1 is a system for displaying or editing ARor VR content.

The information processing device 30, the display device 10, and thecontroller 20 are communicably connected in a wired or wireless mannervia a predetermined network (not illustrated). Note that the informationprocessing system 1 illustrated in FIG. 3 may include a plurality ofdisplay devices 10 and a plurality of information processing devices 30.

The information processing device 30 controls the display device 10 todisplay the mark of the virtual object α at an instruction position thatis a position determined on the basis of a plurality of directionspointed by the user. The information processing device 30 controlsdisplay on the display device 10 using controller information acquiredfrom the controller 20. The information processing device 30 controlsthe display of the display device 10 by using the information regardingthe position and posture of the display device 10 acquired from thedisplay device 10.

<2-1. Configuration of Display Device>

The display device 10 includes a position-posture detection unit 11, alight receiving unit 12, an acceleration sensor 13, a gyro sensor 14,and a display unit 15. The position-posture detection unit 11 detectsthe position and posture of the display device 10 on the basis ofvarious sensor information acquired from sensors included in the displaydevice 10 such as the light receiving unit 12, the acceleration sensor13, and the gyro sensor 14. The position-posture detection unit 11detects various types of information about the position, orientation,inclination, and posture of the display device 10 on the basis of thesensor information. The position-posture detection unit 11 transmitsinformation regarding the position and posture of the display device 10to the information processing device 30. For example, theposition-posture detection unit 11 may be implemented by variousprocessors such as a central processing unit (CPU), a graphicsprocessing unit (GPU), and a field programmable gate array (FPGA).

The display unit 15 is a display that displays various types ofinformation according to the control of the information processingdevice 30. For example, the display device 10 acquires various types ofinformation from the information processing device 30, and displays theacquired information on the display unit 15. The display unit 15displays the mark of the virtual object α at an instruction positiondetermined on the basis of a plurality of directions indicated by theuser X according to the control of the information processing device 30.The display unit 15 displays the content generated by the informationprocessing device 30.

Note that, in a case where the line of sight of the user X is used todesignate the direction, the display device 10 may include aline-of-sight detection unit that detects the line-of-sight position ofthe user X. The line-of-sight detection unit detects the line of sightof the user X by appropriately using various technologies related toline-of-sight detection. As a technique of line-of-sight detection, forexample, a method of detecting a line of sight on the basis of aposition of a moving point of the eye (for example, a pointcorresponding to a moving portion in the eye such as the iris or thepupil) with respect to a reference point (for example, a pointcorresponding to a non-moving portion in the eye such as the innercorner of the eye or corneal reflex) of the eye may be used. Note thatthe detection of the line of sight is not limited to the above, and theline of sight of the user X may be detected using any line-of-sightdetection technique.

<2-2. Configuration of Controller>

The first controller 20A includes a first position-posture detectionunit 21A, a first light receiving unit 22A, a first acceleration sensor23A, and a first gyro sensor 24A. The first position-posture detectionunit 21A detects the position and posture of the first controller 20A onthe basis of sensor information of the first light receiving unit 22A,the first acceleration sensor 23A, the first gyro sensor 24A, and thelike. The first position-posture detection unit 21A detects controllerinformation related to the position, orientation, inclination, andposture of the first controller 20A on the basis of sensor informationof the first light receiving unit 22A, the first acceleration sensor23A, the first gyro sensor 24A, and the like. The first position-posturedetection unit 21A transmits the controller information to theinformation processing device 30. For example, the firstposition-posture detection unit 21A may be implemented by, for example,various processors such as a CPU, a GPU, and an FPGA. Note that in acase where the first controller 20A emits an actual beam, the firstcontroller 20A has a configuration (a light output unit or the like)that emits a laser beam.

The second controller 20B includes a second position-posture detectionunit 21B, a second light receiving unit 22B, a second accelerationsensor 23B, and a second gyro sensor 24B. The second position-posturedetection unit 21B detects the position and posture of the secondcontroller 20B on the basis of sensor information of the second lightreceiving unit 22B, the second acceleration sensor 23B, the second gyrosensor 24B, and the like. The second position-posture detection unit 21Bdetects controller information related to the position, orientation,inclination, and posture of the second controller 20B on the basis ofsensor information of the second light receiving unit 22B, the secondacceleration sensor 23B, the second gyro sensor 24B, and the like. Thesecond position-posture detection unit 21B transmits the controllerinformation to the information processing device 30. For example, thesecond position-posture detection unit 21B may be implemented by, forexample, various processors such as a CPU, a GPU, and an FPGA. In a casewhere the second controller 20B emits an actual beam, the secondcontroller 20B has a configuration (light output unit or the like) thatemits a laser beam.

<2-3. Configuration of Information Processing Device>

The information processing device 30 executes various processes by theCPU 31. FIG. 4 is a diagram illustrating an example of a functionalconfiguration of the information processing device 30. The informationprocessing device 30 illustrated in FIG. 4 includes a communication unit40, a storage unit 50, and a control unit 60. Note that the informationprocessing device 30 may include, for example, an input unit such as akeyboard and a mouse, which receive various operations from anadministrator or the like of the information processing device 30, and adisplay unit for displaying various types of information.

The communication unit 40 is implemented by, for example, an NIC, acommunication circuit, or the like. Then, the communication unit 40 isconnected to a predetermined network (not illustrated) in a wired orwireless manner, and transmits and receives information to and fromother information processing devices such as the controller 20 and thedisplay device 10.

The storage unit 50 is achieved by, for example, a semiconductor memoryelement such as a random access memory (RAM) or a flash memory, or astorage device such as a hard disk or an optical disk. The storage unit50 includes a direction information storage unit 51, an objectinformation storage unit 52, and an intersection information storageunit 53.

The direction information storage unit 51 stores various types ofinformation regarding instructions of directions. FIG. 5 is a diagramillustrating an example of the direction information storage unit 51.The direction information storage unit 51 illustrated in FIG. 5 managesan instruction component 51B, a type 51C, a gripping flag 51D, and thelike in association with each other for each direction ID 51A. Note thatthe direction information storage unit 51 is not limited to the above,and may manage various types of information according to the purpose andcan be appropriately changed.

The direction ID 51A is, for example, information identifying eachdirection pointed by the user X. The instruction component 51B is, forexample, information identifying a component used by the user X to pointto a direction, for example, a device such as the controller 20 or anelement related to the body of the user X. For example, in a case wherea direction is indicated by a line of sight of the user X, a “line ofsight” may be stored in the instruction component 51B. Furthermore, forexample, in a case where the direction is indicated by a finger of theuser X, a “finger” may be stored in the instruction component 51B.

The type 51C is information indicating the type of the instructioncomponent 51B. For example, the type 51C is information indicating atype of a component used by the user to point to a direction, forexample, a device such as the controller 20 or an element related to thebody of the user X. For example, in a case where the direction isinstructed with the controller 20, a “controller”, a “device”, or thelike is stored in the type 51C. For example, in a case where thedirection is instructed with the user's line of sight, the “line ofsight”, a “body”, or the like is stored in the type 51C. Furthermore,for example, in a case where the direction is instructed with the fingerof the user X, the “finger”, the “body”, or the like is stored in thetype 51C.

In the example of FIG. 5 , a direction (direction DG1) identified by adirection ID “DG1” indicates that the instruction component is the firstcontroller 20A that is identified by “20A”. The direction DG1 indicatesthat the type of the first controller 20A is a controller.

Furthermore, a direction (direction DG2) identified by a direction ID“DG2” indicates that the instruction component is the second controller20B that is identified by “20B”. The direction DG2 indicates that thetype of the second controller 20B is a controller.

The object information storage unit 52 stores various types ofinformation regarding the virtual object α. FIG. 6 is a diagramillustrating an example of the object information storage unit 52. Theobject information storage unit 52 illustrated in FIG. 6 manages anobject information 52B, a flag 52C, and the like in association witheach object ID 52A. The flag 52C includes a gravity flag 52D, a grippingflag 52E, and the like. Note that the object information storage unit 52is not limited to the above, and may manage various types of informationaccording to the purpose.

The object ID 52A is information identifying the virtual object α.Furthermore, the object information 52B is information corresponding tothe virtual object α identified by the object ID 52A. Note that, in theexample illustrated in FIG. 6 , the object information 52B isillustrated with an abstract code such as “OINF1”, but various types ofinformation related to the size, shape, and the like of the virtualobject α may be stored. The flag 52C is flag information correspondingto the virtual object α identified by the object ID 52A. The gravityflag 52D is information identifying whether or not the gravity flag isassigned to the virtual object α identified by the object ID 52A. Thegripping flag 52E is information identifying whether or not the grippingflag is assigned to the virtual object α identified by the object ID52A.

In the example of FIG. 6 , the virtual object VO1 identified by theobject ID “VO1” indicates that the object information is “OINF1”. Thevirtual object VO1 indicates that the gravity flag is “1”. That is, thevirtual object VO1 is affected by gravity in determining the arrangementposition. In this case, for example, it is indicated that, when arrangedin the air, the virtual object VO1 is arranged at a position dropped inthe gravity direction from the arranged position.

Furthermore, the virtual object VO41 identified by the object ID “VO41”indicates that the object information is “OINF41”. The virtual objectVO41 indicates that the gravity flag is “0”. That is, it is indicatedthat the virtual object VO41 is not affected by gravity in determiningthe arrangement position. In this case, for example, it is indicatedthat, when arranged in the air, the virtual object VO41 remains at thearranged position.

The intersection information storage unit 53 stores various types ofinformation regarding the intersection. FIG. 7 is a diagram illustratingan example of the intersection information storage unit 53. Theintersection information storage unit 53 illustrated in FIG. 7 managesmaster-slave information 53B in association with each intersection ID53A. The intersection ID 53A is information identifying an intersection.The master-slave information 53B manages a master-slave relationship53C, a master instruction component 53D, and a slave instructioncomponent 53E in association with each other. Note that the intersectioninformation storage unit 53 is not limited to the above, and may storevarious types of information according to the purpose. The master-slaverelationship 53C is information indicating the presence or absence of amaster-slave relationship between instruction components that instructeach direction.

The master instruction component 53D is information indicating that adirection (geometric target) indicated by the instruction component ismaster. In a case where the first controller 20A is the masterinstruction component in the example of FIG. 1 , then “20A” is stored inthe master instruction component 53D. The slave instruction component53E is information indicating that the direction (geometric target)indicated by the instruction component is slave. In a case where thesecond controller 20B is the secondary instruction component in theexample of FIG. 1 , the slave instruction component 53E stores “20B”.

The control unit 60 is implemented by, for example, a central processingunit (CPU), a micro processing unit (MPU), or the like executing aprogram (for example, an information processing program such as aprogram according to the present disclosure) stored inside theinformation processing device 30 with the RAM or the like being a workarea. Furthermore, the control unit 60 is implemented by, for example,an integrated circuit such as an application specific integrated circuit(ASIC) or an FPGA.

<2-4. Functional Configuration of Control Unit>

The control unit 60 includes an acquisition unit 61, a detection unit62, a setting unit 63, a distance determination unit 64, a determinationunit 65, and a display control unit 66, and implements or executes afunction and an action of information processing described below. Notethat the internal configuration of the control unit 60 is not limited tothe configuration illustrated in FIG. 4 , and may be anotherconfiguration as long as information processing to be described later isperformed. Furthermore, the connection relationship of the processingunits included in the control unit 60 is not limited to the connectionrelationship illustrated in FIG. 4 , and may be another connectionrelationship.

The acquisition unit 61 acquires an operation angle that is an angleformed by a main line LN1 in the first direction extending in the depthdirection and an operation line LN3 in the second direction of thehorizontal plane with respect to the first direction. The detection unit62 detects a movement instruction indicating the start of movement ofthe virtual object α. The setting unit 63 sets an intersection betweenthe main line LN1 and the operation line LN3 at a time point when themovement instruction is detected as P0, a reference angle which is anangle formed by the main line LN1 and the operation line LN3 as θ0, anda distance between the main controller 20 and the intersection P0 as D0(see FIG. 12 ).

The acquisition unit 61 acquires an operation angle θ at theintersection of the main line LN1 and the operation line LN3 in responseto a change in the second direction by a sub controller 20. At thistime, an intersection between the main line LN1 and the operation lineLN3 after the change is assumed as P, an operation angle that is anangle formed by the main line LN1 and the operation line LN3 is assumedas θ, and a distance between the main controller 20 and the intersectionP is assumed as DO.

In a case where the acquisition unit 61 acquires the operation angle θin response to the change in the second direction, the distancedetermination unit 64 determines whether or not the distance D1 betweenthe virtual object α on the main line LN1 and the intersection is lessthan a threshold Dth (see FIG. 12 ).

In a case where the distance D1 between the virtual object α and theintersection P is less than the threshold Dth, the display control unit66 causes the display unit 15 to display the virtual object α on themain line LN1 so as to be attracted to the intersection P. At this time,the distance D1 between the virtual object α and the intersection P is“0”.

In a case where the distance D1 between the virtual object α and theintersection P is less than the threshold Dth, the determination unit 65determines whether or not the operation angle θ acquired in response tothe change of the operation line LN3 is equal to or more than thereference angle θ0.

In a case where the operation angle θ is equal to or more than thereference angle θ0, the display control unit 66 moves the virtual objectα to a near side in the depth direction, and causes the display unit 15to display the virtual object α. The display control unit 66 obtains thedistance d between the main controller 20 and the virtual object α bydistance d=(D0+D1)−|P−P0|, but since D1=0, the distance d varies suchthat the virtual object α is on the near side according to the movementamount of |P−P0| corresponding to the change in the operation line LN3(see FIG. 12 ).

In a case where the operation angle θ is not equal to or more than thereference angle θ0, that is, in a case where the operation angle θ isless than the reference angle θ0, the display control unit 66 moves thevirtual object α to a far side in the depth direction, and causes thedisplay unit 15 to display the virtual object α. The display controlunit 66 obtains the distance d between the main controller 20 and thevirtual object α by distance d=(D0+D1)+|P−P0|, but since D1=0, thedistance d varies such that the virtual object α is on the far sideaccording to the movement amount of |P−P0| corresponding to the changein the operation line LN3.

In a case where the distance D1 between the virtual object α and theintersection P is not less than the threshold Dth, that is, in a casewhere the distance D1 is equal to or more than the threshold Dth, thedisplay control unit 66 causes the display unit 15 to display thevirtual object α and the intersection P in a state where the distance D1between the virtual object α and the intersection P is maintained. Atthis time, the distance D1 between the virtual object α and theintersection P is “D1>0”.

The determination unit 65 determines whether or not the operation angleθ acquired in response to the change of the operation line LN3 is equalto or more than the reference angle θ0 while maintaining the distanceD1.

In a case where the operation angle θ is equal to or more than thereference angle θ0, the display control unit 66 moves the virtual objectα and the intersection P to the near side in the depth direction on themain line LN1 while maintaining the distance D1, and causes the displayunit 15 to display the virtual object α and the intersection P. Thedisplay control unit 66 obtains the distance d between the firstcontroller 20A and the virtual object α by distance d=(D0+D1)−|P−P0|,but since D1>0, the distance d varies such that the virtual object is onthe near side according to the movement amount of |P−P0| correspondingto the change in the operation line LN3. Furthermore, the displaycontrol unit 66 arranges the virtual object α on the near side on themain line LN1 of the coordinates obtained by the origin PC0+e (unitvector)×d of the main line, and causes the display unit 15 to displaythe virtual object α.

In a case where the operation angle θ is not equal to or more than thereference angle θ0, that is, in a case where the operation angle θ isless than the reference angle θ0, the display control unit 66 moves thevirtual object α and the intersection P to the far side in the depthdirection on the main line LN1 while maintaining the distance D1, andcauses the display unit 15 to display the virtual object α and theintersection P. The display control unit 66 obtains the distance dbetween the first controller 20A and the virtual object α by distanced=(D0+D1)+|P−P0|, but since D1>0, the distance d varies such that thevirtual object α is on the far side according to the movement amount of|P−P0| corresponding to the change in the operation line LN3.Furthermore, the display control unit 66 arranges the virtual object αon the far side on the main line LN1 of the coordinates obtained by theorigin PC0+e (unit vector)×d of the main line, and causes the displayunit 15 to display the virtual object α.

In the example of FIG. 1 , the display control unit 66 controls thedisplay device 10 to display the mark MK1 of the virtual object α at theinstruction position determined by the user X on the basis of the twodirections indicated by the controller 20. The display control unit 66controls the display device 10 to display the lines LN1 and LN2 on thebasis of the controller information acquired by the acquisition unit 61.

The display control unit 66 controls the display device 10 to displaythe lines LN1 and LN2 on the basis of the controller informationacquired by the acquisition unit 61. The display control unit 66controls display device 10 to display the intersection P. The displaycontrol unit 66 controls the display device 10 to display the mark MK1as illustrated in FIG. 1 at the intersection P as the intersection P.

3. Operation of Information Processing System

Next, operation of the information processing device 30 will bedescribed. As a precondition, the two controllers 20 are used, and theintersection P is generated using lines pointed by the respectivecontrollers 20. The pointing line has a master-slave relationship. Themaster-slave relationship of the pointing line may be determined by anyof priority order, order, and first win of operation. The trigger of thecontroller 20 displaying the main/sub line is referred to as a main/subtrigger. There is a virtual object α in the air (in empty space). Thevirtual object α is displayed in a star shape.

The information processing device 30 includes an intersection generatingprocess, a gripping process of the virtual object α, a moving process ofthe virtual object α, and a releasing process of the virtual object α.

<3-1. Intersection Generating Process>

The intersection generating process is a process of creating anintersection P where the main line LN1 and the operation line LN3intersect. The generating process includes an origin setting process, areference plane setting process, an operation plane setting process, andan intersection setting process.

FIG. 8 is a view illustrating an example of the origin setting process.FIG. 8 illustrates a case where the user X carries the first controller20A as the master instruction component in the right hand as thedominant hand, and carries the second controller 20B as the slaveinstruction component in the left hand. Note that the second controller20B carried by the user X in the left hand may be the master instructioncomponent, and the first controller 20A carried by the user X in theright hand may be the slave instruction component.

As illustrated in FIG. 8 , for example, the origin setting process is aprocess of setting an origin 20A1 of the main line LN1 in the firstdirection pointing in the predetermined space using the first controller20A and an origin 20B1 of the sub line LN2 in the second directionpointing in the predetermined space using the second controller 20B. Theposition of the first controller 20A pointing to the first direction isassumed as the origin 20A1 of the main line LN1, and the position of thesecond controller 20B pointing to the second direction is assumed as theorigin 20B1 of the sub line LN2.

The main line LN1 extends from the first controller 20A along an axispassing through the origin 20A1 of the first controller 20A.Furthermore, the sub line LN2 extends from the second controller 20Balong an axis passing through the origin 20B1 of the second controller20B. For example, the information processing device 30 calculates themain line LN1 on the basis of the position and orientation of the firstcontroller 20A, and calculates the sub line LN2 on the basis of theposition and orientation of the second controller 20B. The informationprocessing device 30 calculates the main line LN1 on the basis of theaxis of the first controller 20A and the origin 20A1, and calculates thesub line LN2 on the basis of the axis of the second controller 20B andthe origin 20B1.

FIG. 9 is a view illustrating an example of the reference plane settingprocess. As illustrated in FIG. 9 , the reference plane setting processis a process of setting a reference plane. A plane formed by the mainline LN1 and the origin 20B1 of the sub line LN2 is assumed as areference plane FC1. The reference plane FC1 is a plane including themain line LN1 and the origin 20B1 of the sub line LN2. In other words,the main line LN1 passes through the reference plane FC1, and the origin20B1 is located in the reference plane FC1. In this manner, thereference plane FC1 is determined by the main line LN1 and the origin20B1 of the sub line LN2. For example, the information processing device30 calculates the reference plane FC1 on the basis of the position ofthe main line LN1 and the position of the origin 20B1. The informationprocessing device 30 calculates the plane including the main line LN1and the origin 20B1 of the sub line LN2 as the reference plane FC1.

Note that the reason for setting the reference plane FC1 is that, forexample, in a case where the ground or floor is fixed to the referenceplane FC1 when the two controllers 20 are operated in a directionperpendicular to the floor (such as right above or right below), theintersection P is to be created from the angle in a direction horizontalto the floor, and thus the intersection P cannot be created at theposition assumed by the user X. Accordingly, the plane formed by themain line LN1 and the origin 20B1 of the sub line LN2 is set as thereference plane FC1.

FIG. 10 is a view illustrating an example of the operation plane settingprocess. As illustrated in FIG. 10 , the operation plane setting processis a process of setting an operation plane FC2. A plane to which thereference plane FC1 is perpendicular with the origin 20B1 of the subline LN2 being the center is assumed as the operation plane FC2. Theoperation plane FC2 is a plane that passes through the origin 20B1 andis orthogonal to the reference plane FC1. Furthermore, the sub line LN2moves on the operation plane FC2. The sub line LN2 rides on theoperation plane FC2. In other words, the sub line LN2 passes through theoperation plane FC2. That is, the operation plane FC2 is a planeincluding the origin 20B1 of the sub line LN2 and orthogonal to thereference plane FC1. The operation plane FC2 is determined by thereference plane FC1 and the origin 20B1 of the sub line LN2. Forexample, the information processing device 30 calculates the operationplane FC2 on the basis of the position of the reference plane FC1, theposition of the sub line LN2, and the position of the origin 20B1. Theinformation processing device 30 calculates a plane orthogonal to thereference plane FC1 and including the sub line LN2 and the origin 20B1as the operation plane FC2. A plane to which the reference plane FC1 isperpendicular with the origin 20B1 of the sub line LN2 being the center,the plane on which the sub line LN2 moves and which includes the subline LN2 and perpendicular to the reference plane FC1, is assumed as theoperation plane FC2 diagram.

FIG. 11 is a view illustrating an example of the intersection settingprocess. As illustrated in FIG. 11 , the intersection setting process isa process of setting a point at which the main line LN1 and theoperation plane FC2 intersect as the intersection P. The point at whichthe main line LN1 and the operation plane FC2 intersect is assumed asthe intersection P. The information processing device 30 calculates theintersection P on the basis of the position of the main line LN1 and theposition of the operation plane FC2. For example, the informationprocessing device 30 calculates the point at which the main line LN1 andthe operation plane FC2 intersect as the intersection P. Note that aline connecting the origin 20B1 of the sub line LN2 and the intersectionP is assumed as the operation line LN3, and an angle formed by theoperation line LN3 and the main line LN1 is assumed as the operationangle θ.

FIG. 12 is a view illustrating an example of a relationship between themain line LN1 and the operation line LN3. The main line LN1 illustratedin FIG. 12 is a line in the first direction pointed by the maincontroller 20. The operation line LN3 is a line in the second directionpointed by the sub controller 20. The distance between the maincontroller 20 on the main line LN1 and the virtual object α is assumedas d, the distance between the main controller 20 on the main line LN1and the intersection P is assumed as DO, and the distance between theintersection P0 on the main line LN1 and the virtual object α is assumedas D1. An angle formed by the main line LN1 and the operation line LN3is assumed as θ0. An intersection between the main line LN1 and theoperation line LN3 at a reference time is assumed as P0, and an angle θ0formed by the main line LN1 and the operation line LN3 is assumed as areference angle. Further, an intersection between the main line LN1 andthe operation line LN3 in a case where the operation line LN3 changesaccording to the operation of the sub controller 20 is assumed as P, andan angle θ formed by the main line LN1 and the operation line LN3 isassumed as an operation angle.

<3-2. Gripping Process of Virtual Object>

The gripping process of a virtual object is a process of gripping thevirtual object α on the main line LN1. FIG. 13 is a view illustrating anexample of the gripping process (without an intersection) of the virtualobject α. In the gripping process illustrated in FIG. 13 , for example,in a case where there is no intersection, when the main trigger ispressed with the main line LN1 abutting on the virtual object α, thevirtual object α is arranged ahead on the main line LN1. Note that themain trigger is, for example, an operation unit of the first controller20A of the main line LN1. Also, in a case where the main trigger is notpressed, the line remains as it is. Further, even in a case where themain trigger is pressed when it is directed to a place where there is novirtual object α, the line remains as it is. Furthermore, also in a casewhere only the sub trigger is pressed, nothing happens. The sub triggeris, for example, an operation unit of the second controller 20B of theoperation line LN3.

On the other hand, FIG. 14 is a view illustrating an example of thegripping process (with an intersection) of the virtual object α. In thegripping process illustrated in FIG. 14 , for example, in a case wherethe intersection P formed by the main line LN1 and the operation lineLN3 is on the main line LN1, when the main trigger is pressed, thevirtual object α is attracted to the intersection P in a case where theintersection P on the main line LN1 and the virtual object α are withina certain distance. On the other hand, in a case where the intersectionP on the main line LN1 and the virtual object α are separated from eachother by more than a certain distance, the virtual object α is notattracted. Further, in a case where the main trigger is not pressed, theline remains as it is. Furthermore, even in a case where only the subtrigger is pressed, nothing happens. In a case where the main and subtriggers are simultaneously pressed, the virtual object α is attractedto the intersection P on the main line LN1. Furthermore, even in a casewhere the main trigger is pressed while the sub trigger is beingpressed, the behavior is the same as the simultaneous pressing of themain and sub triggers.

<3-3. Moving Process of Virtual Object>

As illustrated in FIG. 15 , the moving process of the virtual object αis a process of moving the virtual object α arranged on the main lineLN1 in the depth direction according to the operation of the secondcontroller 20B. In a state where the virtual object α is arranged beyondthe main line LN1 in a state where the main trigger is being pressed,the operation line LN3 is moved on the main line LN1 according to theoperation of the second controller 20B to change the intersection P.When the sub trigger is pressed, it is determined whether or not thedistance D1 between the intersection P and the virtual object α at thattime is less than the threshold Dth.

FIG. 15 is a view illustrating an example of the moving process of thevirtual object α (the distance D1 is less than the threshold Dth). In acase where the distance D1 between the intersection P and the virtualobject α is less than the threshold Dth, that is, in a case where theoperation target is in the vicinity, the virtual object α on the mainline LN1 is attracted to the intersection P as illustrated in FIG. 15 ,and the distance D1 becomes “0”. Then, the operation line LN3 moves onthe main line LN1 according to the operation of the second controller20B, and the intersection P moves according to the operation line LN3after the movement, so that the virtual object α moves on the main lineLN1 according to the movement of the intersection P.

FIG. 16 is views illustrating an example of the moving process of thevirtual object α (the distance D1 is equal to or more than the thresholdDth). In a case where the distance D1 between the intersection P and thevirtual object α is equal to or more than the threshold Dth, that is, ina case where the operation target is at a distant place, the distance D1between the virtual object α and the intersection P is “D1>0” asillustrated in FIG. 16 . Then, the operation line LN3 moves on the mainline LN1 according to the operation of the second controller 20B, theintersection P moves according to the operation line LN3 after themovement, and the virtual object α and the intersection P move on themain line LN1 while maintaining the distance between the virtual objectα and the intersection P. Note that the distance D1 between theintersection P and the virtual object α at the time point when the subtrigger is pressed is held. Since a distant operation target can beoperated at hand, it is possible to perform adjustment while confirminga distance to be finely adjusted.

FIGS. 17 to 19 are flowcharts illustrating an example of processingoperation of the information processing device 30 related to the movingprocess. The control unit 60 designates one controller 20 among theplurality of controllers 20 (Step S11), and acquires the orientation andposition of the designated controller 20 (Step S12). The control unit 60determines whether or not the virtual object α in the predeterminedspace is gripped by the designated controller 20 (Step S13). In a casewhere the virtual object α in the predetermined space is gripped (StepS13: Yes), the control unit 60 sets the gripping flag of the designatedcontroller 20 to “1” (Step S14) and determines whether or not there isan undesignated controller 20 (Step S15). The control unit 60 stores “1”in the gripping flag corresponding to the direction ID for identifyingthe line LN of the designated controller 20 stored in the directioninformation storage unit 51.

In a case where there is an undesignated controller 20 (Step S15: Yes),the control unit 60 proceeds to Step S11 to designate the controller 20.Furthermore, in a case where the designated controller 20 does not gripthe virtual object α (Step S13: No), the control unit 60 proceeds toStep S15 to determine whether or not there is an undesignated controller20.

In a case where there is an undesignated controller 20 (Step S15: Yes),the control unit 60 re-designates one controller 20 among the pluralityof designated controllers 20 (Step S16). The control unit 60 determineswhether or not the gripping flag of the re-designated controller 20 is“1” (Step S17). The control unit 60 refers to the gripping flagcorresponding to the direction ID for identifying the line LN of there-designated controller 20 stored in the direction information storageunit 51, and determines whether or not the gripping flag is “1”. In acase where the gripping flag of the re-designated controller 20 is “1”(Step S17: Yes), the control unit 60 determines whether or not thetrigger of the re-designated controller 20 is being pressed (Step S23).In a case where the trigger of the re-designated controller 20 is beingpressed (Step S23: Yes), the control unit 60 determines whether or notthere is a controller 20 that has not been re-designated yet among theplurality of designated controllers 20 (Step S18). In a case where thereis no controller 20 that has not been re-designated yet (Step S18: No),the control unit 60 proceeds to M1 illustrated in FIG. 18 . Furthermore,in a case where the trigger of the re-designated controller 20 is notbeing pressed (Step S23: No), the control unit 60 releases the virtualobject α from the re-designated controller 20 (Step S24), sets thegripping flag of the re-designated controller 20 to “0” (Step S25), andproceeds to Step S18.

In a case where there is a controller 20 that has not been re-designatedyet (Step S18: Yes), the control unit 60 proceeds to Step S16 tore-designate the controller 20.

In a case where the gripping flag of the re-designated controller 20 isnot “1” (Step S17: No), the control unit 60 determines whether or not aline in the direction pointed by the re-designated controller 20 pointsat the virtual object α in the predetermined space (Step S19). In a casewhere the virtual object α is pointed (Step S19: Yes), the control unit60 determines whether or not the trigger of the re-designated controller20 pointing at the virtual object α in the predetermined space is beingpressed (Step S20). Note that the control unit 60 determines, forexample, whether or not the trigger of the first controller 20A is beingpressed in a state where the first controller 20A points at the virtualobject α.

In a case where the trigger of the re-designated controller 20 is beingpressed (Step S20: Yes), the control unit 60 causes the re-designatedcontroller 20 to hold the virtual object α (Step S21), sets the grippingflag of the re-designated controller 20 to “1” (Step S22), and proceedsto Step S18 to determine whether or not there is a controller 20 thathas not been re-designated yet.

In a case where the virtual object α is not pointed (Step S19: No) or ina case where the trigger of the re-designated controller 20 is not beingpressed (Step S20: No), the control unit 60 proceeds to Step S18 todetermine whether or not there is a controller 20 that has not beenre-designated yet.

In M1 illustrated in FIG. 18 , the control unit 60 determines whether ornot there are two controllers 20 (Step S31). In a case where the numberof the controllers 20 is not two (Step S31: No), the control unit 60determines whether or not there is one controller 20 (Step S32). In acase where there is one controller 20 (Step S32: Yes), the control unit60 proceeds to M2 illustrated in FIG. 17 . Furthermore, in a case wherethe number of the controllers 20 is not one (Step S32: No), the controlunit 60 ends the processing operation illustrated in FIG. 18 .

In a case where there are two controllers 20 (Step S31: Yes), thecontrol unit 60 determines whether or not the gripping flags of the twocontrollers 20 are “0” (Step S33). In a case where both the grippingflags of the two controllers 20 are “0” (Step S33: Yes), the controlunit 60 proceeds to M2 illustrated in FIG. 17 .

In a case where the gripping flags of the two controllers 20 are not“0”, the control unit 60 determines whether or not the gripping flags ofthe two controllers 20 are “1” (Step S34). In a case where the grippingflags of the two controllers 20 are not “1” (Step S34: No), that is, thecontrol unit 60 determines that the gripping flag of one of thecontrollers 20 is “1”, and sets the controller 20 having the grippingflag “1” as the main controller 20 (Step S35). Note that the controlunit 60 changes the master-slave relationship 53C stored in theintersection information storage unit 53 to “present”, stores “20A” inthe master instruction component 53D, and stores “20B” in the slaveinstruction component 53E. The control unit 60 sets the distance D1between the intersection P on the main line LN1 of the main controller20 and the virtual object α to “−1” (Step S36), and proceeds to M2illustrated in FIG. 17 .

In a case where the gripping flags of the two controllers 20 are “1”(Step S34: Yes), the control unit 60 determines whether or not there isan intersection on the main line LN1 at which two lines of thecontrollers 20, that is, the main line LN1 the operation line LN3intersect (Step S37). In a case where there is an intersection on themain line LN1 (Step S37: Yes), the control unit 60 determines whether ornot the distance D1 between the intersection and the virtual object α is“0” or more (Step S38).

In a case where the distance D1 between the intersection and the virtualobject α is “0” or more (Step S38: Yes), the distance determination unit64 in the control unit 60 determines whether or not the distance D1 isequal to or less than the threshold Dth (Step S39). In a case where thedistance D1 is equal to or less than the threshold Dth (Step S39: Yes),the control unit 60 sets “0” to the distance D1 by the intersectionbeing attracted to the position of the virtual object α (Step S40).

The setting unit 63 in the control unit 60 sets the intersectioncoordinates of the main line LN1 and the operation line LN3 to P0 (StepS41), sets the angle formed by the main line LN1 and the operation lineLN3 to θ0 as the reference angle (Step S42), sets the distance betweenthe main controller 20 and the intersection to DO (Step S43), andproceeds to M2 illustrated in FIG. 17 .

Furthermore, in a case where the distance D1 is not equal to or lessthan the threshold Dth (Step S39: No), the setting unit 63 in thecontrol unit 60 sets the distance between the intersection and thevirtual object α to D1 (Step S44), and proceeds to Step S41 to set theintersection coordinates of the main line LN1 and the operation line LN3to P0.

Furthermore, in a case where the distance D1 between the intersectionand the virtual object α is not “0” or more (Step S38: No), the controlunit 60 proceeds to M3 illustrated in FIG. 19 .

At M3 illustrated in FIG. 19 , the control unit 60 sets the intersectioncoordinates of the main line LN1 and the operation line LN3 to P (StepS51), and sets the angle formed by the main line LN1 and the operationline LN3 after movement to 0 as an operation angle (Step S52). Thedetermination unit 65 in the control unit 60 determines whether or notthe operation angle θ is equal to or less than the reference angle θ0(Step S53).

In a case where the operation angle θ is equal to or less than thereference angle θ0 (Step S53: Yes), the display control unit 66 in thecontrol unit 60 calculates the distance d between the main controller 20and the virtual object α by (D0+D1)+|P−P0| (Step S54). Note that, in acase where the distance D1 is equal to or less than the threshold Dth,since D1=0, the distance from the main controller 20 to the virtualobject α on the main line LN1 becomes long by the movement amount of|P−P0| according to the change in the operation line LN3. That is, thevirtual object α in the vicinity moves to the far side on the main lineLN1. On the other hand, in a case where the distance D1 is not equal toor less than the threshold Dth, since D1 is an actual measurement value,the distance to the virtual object α on the main line LN1 becomes longby the movement amount of |P−P0| according to the change in theoperation line LN3 while maintaining the distance between theintersection and the virtual object α. That is, the distant virtualobject α moves to the far side on the main line LN1.

Moreover, in a case where the operation angle θ is not equal to or lessthan the reference angle θ0 (Step S53: No), the display control unit 66in the control unit 60 calculates the distance d between the maincontroller 20 and the virtual object α by (D0+D1)−|P−P0| (Step S55).Note that, in a case where the distance D1 is equal to or less than thethreshold Dth, since D1=0, the distance from the main controller 20 tothe virtual object α on the main line LN1 becomes short by the movementamount of |P−P0| according to the change in the operation line LN3. Thatis, the virtual object α in the vicinity moves to the near side on themain line LN1. On the other hand, in a case where the distance D1 is notequal to or less than the threshold Dth, since D1 is an actualmeasurement value, the distance to the virtual object α on the main lineLN1 becomes short by the movement amount of |P−P0| according to thechange in the operation line LN3 while maintaining the distance betweenthe intersection and the virtual object α. That is, the distant virtualobject α moves to the near side on the main line LN1.

Then, the control unit 60 displays the virtual object α at the positionof the PC0+unit vector e×d by using the distance d between the maincontroller 20 and the virtual object a calculated in Step S54 or StepS55 (Step S56), and proceeds to M2 illustrated in FIG. 17 .Consequently, the virtual object α is displayed at a position obtainedby adding the distance multiplied by the unit vector e and the distanced from the origin coordinate PC0 of the main controller 20.

<3-4. Releasing Process of Virtual Object>

The releasing process of the virtual object α is a process of arrangingthe position of the virtual object α adjusted on the main line LN1. Theposition of the virtual object α is adjusted on the main line LN1 inresponse to the change operation of the operation line LN3 while themain trigger is being pressed. Further, when the main trigger isreleased, the virtual object α at the current position on the main lineLN1 is arranged. Furthermore, when the sub trigger is released while themain trigger is being pressed, the virtual object α is fixed to theposition of the virtual object α at the time point when the sub triggeris released. Furthermore, when the main trigger is released while thesub trigger is being pressed, the virtual object α is fixed to thecurrent position of the virtual object α at the time point when the maintrigger is released.

4. Effects of Embodiment

In a case where the distance D1 is equal to or more than the thresholdDth, the control unit 60 determines whether or not the operation angle θacquired in response to the change of the operation line LN3 is equal toor more than the reference angle θ0 while maintaining the distance D1.In a case where the operation angle θ is equal to or more than thereference angle θ0, the control unit 60 moves the virtual object α andthe intersection P to the near side in the depth direction on the mainline LN1 while maintaining the distance D1, and causes the display unit15 to display the virtual object α and the intersection P. At this time,the control unit 60 causes the virtual object α to be displayed on themain line LN1 of the coordinates (coordinates on the near side) obtainedby the origin PC0+e (unit vector)×d of the main line LN1. Consequently,while maintaining the distance between the virtual object α and theintersection, the position of the distant virtual object α can be finelyadjusted to the near side at a hand position where the operation iseasy.

In a case where the operation angle θ is less than the reference angleθ0, the control unit 60 moves the virtual object α and the intersectionP to the depth side in the depth direction on the main line LN1 whilemaintaining the distance D1, and causes the display unit 15 to displaythe virtual object α and the intersection P. At this time, the controlunit 60 causes the virtual object α to be displayed on the main line LN1of the coordinates (coordinates on the far side) obtained by the originPC0+e (unit vector)×d of the main line LN1. Consequently, the positionof the distant virtual object α can be finely adjusted to the far sideat the hand position where the operation is easy while the distancebetween the virtual object α and the intersection is secured.

The virtual object α can be moved by the movement distance while thetrigger is being pressed by using a metaphor of holding and releasingthe virtual object α. Furthermore, the virtual object α can be movedeither far or near depending on the moving direction of the virtualobject α.

In addition, each trigger of the two controllers 20 can be used tofinely adjust the position of a distant intersection or the virtualobject α so as to allow operating at hand. Moreover, the position of thedistant virtual object α can be adjusted by a series of movementswithout re-holding or putting down the controller 20.

5. Modification Example

Furthermore, environment recognition is performed, and a target that canbe operated is limited or determined in advance. In a case where theoperation target is limited, when the line approaches or touches thevirtual object α, it may change to an expression or state of attractingor gripping so that the user can easily understand the operation target.In a case of operating a distant virtual object α, it is difficult tounderstand the situation, positional relationship, and the like in thevicinity of the virtual object α as the operation target, and thus acamera image in the vicinity of the operation target unit may bepresented together as a monitor screen (another window) near theoperation unit at hand. Alternatively, the operator may virtually movenear the operation target and operate the virtual object α.

<5-1. Other Releasing Processes of Virtual Object>

In a case where the virtual object α is separated from the line, it maybe executed by an operation accompanied by a rapid change inacceleration, such as shaking, swinging, or throwing the controller 20,so that the user can easily image that the virtual object α isseparated.

<5-2. Other Gripping Processes of Virtual Object>

For example, the intersection generating process and the grippingprocess of the virtual object α may be executed according to, forexample, a trigger press, a specific gesture (examples include puttinghands together, sticking out, and the like), or a context (when theapplication enters a selection mode). Furthermore, termination of theintersection creation process or the virtual object may be executed, forexample, in a timeout.

<5-3. Other Moving Processes of Virtual Object>

The intersection position and the position of the virtual object α areadjusted by two lines, but one line may be fixed in a direction at acertain time point (for example, this may be specified by uttering “now”or the like in a voice). Different modals such as a line of sight andfinger pointing may be combined, and an operation by a plurality ofpersons, such as operating two lines by two persons, may be performed.

Further, when fine adjustment is performed at hand, for example,feedback indicating a sense of distance by sound, vibration, or the likemay be input every specific distance in units of 10 cm. The distancefrom the user position to the virtual object α may be read aloud.

Furthermore, in the present embodiment, the intersection position may bechanged by an operation of sending or returning the intersectionposition by fixing the operation angle as the reference angle withoutchanging the angle, for example, front-back movement or left-rightmovement of the hand, and the position of the virtual object α may bechanged in conjunction. FIG. 20 is a view illustrating an example of themovement position of the virtual object α at a time of forward movementof the operation line LN3. As illustrated in FIG. 20 , in a case wherethe operation line LN3 is moved forward, the virtual object α on themain line LN1 is moved forward (far side). On the other hand, in a casewhere the operation line LN3 is moved backward, the virtual object α onthe main line LN1 is moved backward (near side).

FIG. 21 is a view illustrating an example of the movement position ofthe virtual object α at a time of leftward movement of the operationline LN3. As illustrated in FIG. 21 , in a case where the operation lineLN3 is moved leftward, the virtual object α on the main line LN1 ismoved forward (far side). On the other hand, in a case where theoperation line LN3 is moved rightward, the virtual object α on the mainline LN1 is moved backward (near side).

The case has been exemplified in which, in a case where the operationangle θ is equal to or more than the reference angle θ0, the controlunit 60 of the present embodiment causes the virtual object α and theintersection P to move to the near side in the depth direction on themain line LN1 and be displayed on the display unit 15 while maintainingthe distance D1. However, in a case where the operation angle θ is equalto or more than the reference angle θ0, the control unit 60 moves thevirtual object α and the intersection P to the far side in the depthdirection on the main line LN1 while maintaining the distance D1, andcauses the display unit 15 to display the virtual object α and theintersection P. Furthermore, in a case where the operation angle θ isless than the reference angle θ0, the control unit 60 may move thevirtual object α and the intersection P to the near side in the depthdirection on the main line LN1 while maintaining the distance D1, andcause the display unit 15 to display the virtual object α and theintersection P, which can be appropriately changed.

<5-4. Other Instruction Components>

Although the first controller 20A and the second controller 20B havebeen exemplified as the instruction components, another device, the body(hand or eye) of the user or the like may be used instead of the devicesuch as the controller 20. For example, the controller 20 and the lineof sight of the eyes of the user X may be the instruction component.

The instruction component is not limited to the above, and may bevarious elements such as a palm, an arm, and a front of a face or ahead. That is, ones that emit a line include various objects capable ofindicating a direction, such as a controller, a finger, a hand, a palm,an arm, a line of sight, and a front of a face or head.

<5-5. Other Display Forms of Virtual Object>

For example, in order to facilitate understanding of weight andcharacteristics of the virtual object α, movement of a line orexpression of the virtual object α indicating inertia or reaction forcemay be performed. In order to make it easy to understand that thevirtual object α overlaps or collides with another virtual object α oran object in the real world, an expression in which a line or thevirtual object α bends or flicks may be added. In a case where thevirtual object α cannot move deeper in the depth direction, anexpression in which the virtual object α collides and is pressed may beincorporated in the line or the virtual object α.

In a case where the position of the virtual object α is moving, theinformation processing device 30 controls the display unit 15 to changeand display the display mode of the virtual object α. For example, in acase where the virtual object α is moved and arranged, the informationprocessing device 30 may weaken the display of the virtual object αbeing moved. For example, the information processing device 30 weakensthe display of the moving virtual object α by increasing thetransmittance of the virtual object α being moved. As described above,when the virtual object α is moved and arranged, by weakening thedisplay of the virtual object α being moved, the user X can move thevirtual object α while confirming the arrangement and positionalrelationship with the object in the real world or the virtual object αarranged around the object.

Furthermore, for example, in a case where the virtual object α is movedand arranged, the information processing device 30 may enhance thedisplay of the virtual object α being moved. In this manner, byenhancing the display of the virtual object α being moved, the user Xcan make the virtual object α being moved conspicuous (enhancevisibility) among similar virtual objects a or easily arranged to theback.

Furthermore, as described above, various display modes may be usedduring the movement of the virtual object α. For example, theinformation processing device 30 may cause the virtual object α to bedisplayed as it is even while moving. The information processing device30 displays the virtual object α as it is in a case where it is desiredto arrange the virtual object α while confirming the arrangement andpositional relationship with the object in the real world or the virtualobject α arranged around the object.

Furthermore, for example, the information processing device 30 mayweaken the display of the virtual object α being moved. For example, theinformation processing device 30 displays only the outline of thevirtual object α or makes it translucent. The information processingdevice 30 displays the outline of the virtual object α or makes thevirtual object α translucent in a case where it is desired to performtrajectory and position adjustment during movement while confirming thearrangement and positional relationship with the object in the realworld or the virtual object α arranged around the object. For example,the information processing device 30 may turn off the display of thevirtual object α. In this case, the information processing device 30 maycause only the intersection to be displayed. In a case where it isdesired to emphasis the trajectory and position adjustment duringmovement and make it easier to see, the information processing device 30deletes the display of the virtual object α.

Furthermore, for example, the information processing device 30 mayenhance the display of the virtual object α being moved. The informationprocessing device 30 may enhance the hue or increase the luminancevalue. The information processing device 30 may be combined with anadditional display such as an icon. In a case where similar objects arearranged, the information processing device 30 highlights the selectedvirtual object α for easy recognizing. Furthermore, in a case where itis desired to arrange the virtual object α at the back of a place wherethe virtual objects a are arranged, the information processing device 30suppresses the color tone of the other virtual objects and increases thetransmittance, so that the arrangement at the back of the virtualobjects becomes easy.

<5-6. Other Master-Slave Relationships>

Furthermore, even in a case where there is no functional differencebetween the two lines, the two lines may have a master-slaverelationship. The information processing device 30 may determine themaster-slave relationship between the instruction components byappropriately using various types of information. Furthermore, theinformation processing device 30 may change the color density, shape,and the like of each line in order to indicate the master-slaverelationship.

The information processing device 30 may determine an instructioncomponent estimated to be preferentially used as a main instructioncomponent (master instruction component). For example, the informationprocessing device 30 sets an instruction component corresponding to theuser's right hand as the master instruction component. Furthermore, forexample, the information processing device 30 may set one having anobject (device) as the master instruction component. For example, in acase of carrying the object (device) only in one hand, the informationprocessing device 30 may set this device as the master instructioncomponent.

The information processing device 30 may determine the masterinstruction component according to a predetermined order. Theinformation processing device 30 may determine the master instructioncomponent according to the order of bringing into a beam state. Forexample, the information processing device 30 may determine theinstruction component that has been first brought into a beam state asthe master instruction component.

The information processing device 30 may determine the masterinstruction component according to the way of movement. For example, theinformation processing device 30 may determine an instruction componentthat has been moved largely or moved earlier as the master instructioncomponent.

<5-7. Other Display Forms of Lines>

Furthermore, the representation of the lines may also be of varioustargets. For example, when it becomes accustomed to the operation, thedisplay of the line becomes unnecessary, and thus only the intersectionmay be displayed. Furthermore, for example, the information processingdevice 30 may express a line so that it is easy to create anintersection when two lines are separated. For example, the informationprocessing device 30 may increase the thickness of the line.

Note that the present invention is not limited to the above, and variousdisplay modes may be used. Although the operation in a case where thereis one user has been described in the above-described example, aplurality of users (a plurality of persons) may wear the display device10 such as an AR device, a VR device, or an MR device, and operate by aplurality of persons while viewing the same video. In this case, whenone person is operating the virtual object α, the other person canadjust the position.

<5-8. Other Geometric Targets>

Note that, as described above, the geometric target is not limited to aplane (operation plane) and may be a line. In the present embodiment, acase where the intersection is created by crossing the operation lineLN3 and the main line LN1 has been exemplified, but the intersection maybe created by crossing, not the operation line, but the sub line LN2 andthe main line LN1, which can be appropriately changed.

The information processing device 30 of the present embodiment may beimplemented by a dedicated computer system or a general-purpose computersystem.

6. Hardware Configuration

The information device such as the information processing device 100according to each of the above-described embodiments and modificationexamples is implemented by, for example, a computer 100 having aconfiguration as illustrated in FIG. 22 . FIG. 22 is a hardwareconfiguration diagram illustrating an example of the computer 100 thatimplements the functions of the information processing device such asthe information processing device 30. Hereinafter, the informationprocessing device 30 according to the embodiment will be described as anexample. The computer 100 includes a CPU 110, a RAM 120, a read onlymemory (ROM) 130, a hard disk drive (HDD) 140, a communication interface150, and an input-output interface 160. Each unit of the computer 100 isconnected by a bus 170.

The CPU 110 operates on the basis of a program stored in the ROM 130 orthe HDD 140, and controls each unit. For example, the CPU 110 develops aprogram stored in the ROM 130 or the HDD 140 in the RAM 120, andexecutes processing corresponding to various programs.

The ROM 130 stores a boot program such as a basic input output system(BIOS) executed by the CPU 110 when the computer 100 is activated, aprogram depending on hardware of the computer 100, and the like.

The HDD 140 is a computer-readable recording medium that non-transientlyrecords a program executed by the CPU 110, data used by the program, andthe like. Specifically, the HDD 140 is a recording medium that recordsan information processing program according to the present disclosure asan example of program data 145.

The communication interface 150 is an interface for the computer 100 toconnect to an external network 155 (for example, the Internet). Forexample, the CPU 110 receives data from another device or transmits datagenerated by the CPU 110 to another device via the communicationinterface 150.

The input-output interface 160 is an interface for connecting aninput-output device 165 and the computer 100. For example, the CPU 110receives data from an input device such as a keyboard and a mouse viathe input-output interface 160. Further, the CPU 110 transmits data toan output device such as a display, a speaker, or a printer via theinput-output interface 160. Furthermore, the input-output interface 160may function as a media interface that reads a program or the likerecorded in a predetermined recording medium. The medium is, forexample, an optical recording medium such as a digital versatile disc(DVD) or a phase change rewritable disk (PD), a magneto-opticalrecording medium such as a magneto-optical disk (MO), a tape medium, amagnetic recording medium, a semiconductor memory, or the like.

For example, in a case where the computer 100 functions as theinformation processing device 30 according to the embodiment, the CPU110 of the computer 100 implements the functions of the control unit 60and the like by executing the information processing program loaded onthe RAM 120. Further, the HDD 140 stores an information processingprogram according to the present disclosure and data in the storage unit50. Note that the CPU 110 reads the program data 145 from the HDD 140and executes the program data 145, but as another example, theseprograms may be acquired from another device via the external network155.

For example, a program for executing the above-described operation (forexample, the moving process of the virtual object α) is stored in acomputer-readable recording medium such as an optical disk, asemiconductor memory, a magnetic tape, or a flexible disk anddistributed. Then, for example, by installing the program in a computerand executing the above-described processing, the information processingdevice 30 can be configured.

Further, the program may be stored in a storage device included inanother information processing device on a network such as the Internetso that the program can be downloaded to a computer. Furthermore, theabove-described functions may be implemented by cooperation of anoperating system (OS) and application software. In this case, a portionother than the OS may be stored in a medium and distributed, or aportion other than the OS may be stored in a server device, anddownloading to a computer, or the like can be performed.

Furthermore, among the processes described in the above embodiments, allor part of the processes described as being performed automatically canbe performed manually, or all or part of the processes described asbeing performed manually can be performed automatically by a publiclyknown method. Further, the processing procedure, specific name, andinformation including various data and parameters illustrated in thedocument and the drawings can be arbitrarily changed unless otherwisespecified. For example, the various types of information illustrated ineach figure are not limited to the illustrated information. Furthermore,in the above embodiments, there is a portion where a specific value ispresented and described, but the value is not limited to the example,and another value may be used.

Further, each component of each device illustrated in the drawings isfunctionally conceptual, and is not necessarily physically configured asillustrated in the drawings. That is, a specific form of distributionand integration of each device is not limited to the illustrated form,and all or a part thereof can be functionally or physically distributedand integrated in any unit according to various loads, usage conditions,and the like.

Further, the above-described embodiments can be appropriately combinedin a region in which the processing contents do not contradict eachother. Furthermore, the order of respective steps illustrated in theflowcharts and the sequence diagrams of the above-described embodimentscan be changed as appropriate.

Furthermore, for example, the present embodiment can be implemented asany configuration constituting a device or a system, for example, aprocessor as a system large scale integration (LSI) or the like, amodule using a plurality of processors or the like, a unit using aplurality of modules or the like, a set obtained by further adding otherfunctions to a unit, or the like (that is, a configuration of a part ofthe device).

Note that, in the present embodiment, the system means a set of aplurality of components (devices, modules (parts), and the like), and itdoes not matter whether or not all the components are in the samehousing. Therefore, a plurality of devices housed in separate housingsand connected via a network and one device in which a plurality ofmodules is housed in one housing are both systems.

Furthermore, the present embodiment can employ, for example, aconfiguration of cloud computing in which at least one function (forexample, the acquisition unit 61, the detection unit 62, the settingunit 63, the distance determination unit 64, the determination unit 65,or the display control unit 66) is shared and processed in cooperationby a plurality of devices via a network.

7. Conclusion

As described above, an information processing device according to anembodiment of the present disclosure includes an acquisition unit thatacquires an operation angle that is an angle formed by a first directionin a predetermined space pointed by a user and a second direction in thepredetermined space pointed by the user, a setting unit that sets, as areference angle, the operation angle acquired at a time point when aninstruction to start moving a virtual object on a line extending in thefirst direction is detected, a determination unit that determineswhether or not the operation angle acquired by the acquisition unit inresponse to a change in the second direction is equal to or more thanthe reference angle, and a control unit that controls a display deviceto move the virtual object in a depth direction on the line in the firstdirection and display the virtual object while maintaining a distancebetween an intersection where the first direction and the seconddirection intersect and the virtual object on the basis of adetermination result of the determination unit. Consequently, whilemaintaining the distance between the virtual object α and theintersection, the position of the distant virtual object α can be finelyadjusted at the hand position where the operation is easy.

Although the embodiments of the present disclosure have been describedabove, the technical scope of the present disclosure is not limited tothe above-described embodiments as it is, and various modifications canbe made without departing from the gist of the present disclosure.Furthermore, components of different embodiments and modificationexamples may be appropriately combined.

Furthermore, the effects in the embodiments described in the presentdescription are merely examples and are not limited, and other effectsmay be provided.

Note that the present technology can also have the followingconfigurations.

(1)

An information processing device including:

an acquisition unit that acquires an operation angle that is an angleformed by a first direction in a predetermined space pointed by a userand a second direction in the predetermined space pointed by the user;

a setting unit that sets, as a reference angle, the operation angleacquired at a time point when an instruction to start moving a virtualobject on a line extending in the first direction is detected;

a determination unit that determines whether or not the operation angleacquired by the acquisition unit in response to a change in the seconddirection is equal to or more than the reference angle; and

a control unit that controls a display device to move the virtual objectin a depth direction on the line in the first direction and display thevirtual object while maintaining a distance between an intersectionwhere the first direction and the second direction intersect and thevirtual object on a basis of a determination result of the determinationunit.

(2)

The information processing device according to (1), wherein

the control unit

controls the display device to move the virtual object to a near side inthe depth direction on the line in the first direction and display thevirtual object while maintaining the distance between the intersectionand the virtual object in a case where the operation angle is equal toor more than the reference angle in the determination unit.

(3)

The information processing device according to (1) or (2), wherein

the control unit

controls the display device to move the virtual object to a far side inthe depth direction on the line in the first direction and display thevirtual object while maintaining the distance between the intersectionand the virtual object in a case where the operation angle is not equalto or more than the reference angle in the determination unit.

(4)

The information processing device according to (1), wherein

the control unit

controls the display device to move the virtual object to the far sidein the depth direction on the line in the first direction and displaythe virtual object while maintaining the distance between theintersection and the virtual object in a case where the operation angleis equal to or more than the reference angle in the determination unit.

(5)

The information processing device according to (1) or (2), wherein

the control unit

controls the display device to move the virtual object to the near sidein the depth direction on the line in the first direction and displaythe virtual object while maintaining the distance between theintersection and the virtual object in a case where the operation angleis not equal to or more than the reference angle in the determinationunit.

(6)

The information processing device according to any one of (1) to (5),including:

a first distance determination unit that determines whether or not thedistance between the virtual object and the intersection is equal to ormore than a threshold, wherein

the determination unit

determines whether or not the operation angle is equal to or more thanthe reference angle in response to the change in the second direction ina case where the distance between the virtual object and theintersection is equal to or more than the threshold.

(7)

The information processing device according to (6), wherein

the control unit

controls the display device in such a manner that the intersection isattracted to a position of the virtual object in a case where thedistance between the virtual object and the intersection is less thanthe threshold.

(8)

The information processing device according to any one of (1) to (7),wherein

the second direction is,

on an operation plane perpendicular to a reference plane determined byan origin of a third direction in the predetermined space pointed by theuser and the first direction, a direction on an operation line throughwhich an intersection where the third direction and the first directionintersect and the origin of the third direction pass.

(9)

The information processing device according to (1), wherein

the control unit

controls the display device to move the virtual object that is anoperation target in the depth direction on the line in the firstdirection and display the virtual object while maintaining the distancebetween the intersection where the first direction and the seconddirection intersect and the virtual object, and controls the displaydevice to display an image near the operation target in another window,on a basis of the determination result of the determination unit.

(10)

An information processing method including, by an information processingdevice:

acquiring an operation angle that is an angle formed by a firstdirection in a predetermined space pointed by a user and a seconddirection in the predetermined space pointed by the user;

setting, as a reference angle, the operation angle acquired at a timepoint when an instruction to start moving a virtual object on a lineextending in the first direction is detected;

determining whether or not the operation angle acquired in response to achange in the second direction is equal to or more than the referenceangle; and

controlling a display device to move the virtual object in a depthdirection on the line in the first direction and display the virtualobject while maintaining a distance between an intersection where thefirst direction and the second direction intersect and the virtualobject on a basis of a determination result.

(11)

An information processing program causing a computer to executeprocessing including:

acquiring an operation angle that is an angle formed by a firstdirection in a predetermined space pointed by a user and a seconddirection in the predetermined space pointed by the user;

setting, as a reference angle, the operation angle acquired at a timepoint when an instruction to start moving a virtual object on a lineextending in the first direction is detected;

determining whether or not the operation angle acquired in response to achange in the second direction is equal to or more than the referenceangle; and

controlling a display device to move the virtual object in a depthdirection on the line in the first direction and display the virtualobject while maintaining a distance between an intersection where thefirst direction and the second direction intersect and the virtualobject on a basis of a determination result.

REFERENCE SIGNS LIST

-   -   1 INFORMATION PROCESSING SYSTEM    -   10 DISPLAY DEVICE    -   15 DISPLAY UNIT    -   20 CONTROLLER    -   20A FIRST CONTROLLER    -   20B SECOND CONTROLLER    -   30 INFORMATION PROCESSING DEVICE    -   60 CONTROL UNIT    -   61 ACQUISITION UNIT    -   62 DETECTION UNIT    -   63 SETTING UNIT    -   64 DISTANCE DETERMINATION UNIT    -   65 DETERMINATION UNIT    -   66 DISPLAY CONTROL UNIT

1. An information processing device including: an acquisition unit thatacquires an operation angle that is an angle formed by a first directionin a predetermined space pointed by a user and a second direction in thepredetermined space pointed by the user; a setting unit that sets, as areference angle, the operation angle acquired at a time point when aninstruction to start moving a virtual object on a line extending in thefirst direction is detected; a determination unit that determineswhether or not the operation angle acquired by the acquisition unit inresponse to a change in the second direction is equal to or more thanthe reference angle; and a control unit that controls a display deviceto move the virtual object in a depth direction on the line in the firstdirection and display the virtual object while maintaining a distancebetween an intersection where the first direction and the seconddirection intersect and the virtual object on a basis of a determinationresult of the determination unit.
 2. The information processing deviceaccording to claim 1, wherein the control unit controls the displaydevice to move the virtual object to a near side in the depth directionon the line in the first direction and display the virtual object whilemaintaining the distance between the intersection and the virtual objectin a case where the operation angle is equal to or more than thereference angle in the determination unit.
 3. The information processingdevice according to claim 1, wherein the control unit controls thedisplay device to move the virtual object to a far side in the depthdirection on the line in the first direction and display the virtualobject while maintaining the distance between the intersection and thevirtual object in a case where the operation angle is not equal to ormore than the reference angle in the determination unit.
 4. Theinformation processing device according to claim 1, wherein the controlunit controls the display device to move the virtual object to the farside in the depth direction on the line in the first direction anddisplay the virtual object while maintaining the distance between theintersection and the virtual object in a case where the operation angleis equal to or more than the reference angle in the determination unit.5. The information processing device according to claim 1, wherein thecontrol unit controls the display device to move the virtual object tothe near side in the depth direction on the line in the first directionand display the virtual object while maintaining the distance betweenthe intersection and the virtual object in a case where the operationangle is not equal to or more than the reference angle in thedetermination unit.
 6. The information processing device according toclaim 1, including: a first distance determination unit that determineswhether or not the distance between the virtual object and theintersection is equal to or more than a threshold, wherein thedetermination unit determines whether or not the operation angle isequal to or more than the reference angle in response to the change inthe second direction in a case where the distance between the virtualobject and the intersection is equal to or more than the threshold. 7.The information processing device according to claim 6, wherein thecontrol unit controls the display device in such a manner that theintersection is attracted to a position of the virtual object in a casewhere the distance between the virtual object and the intersection isless than the threshold.
 8. The information processing device accordingto claim 1, wherein the second direction is, on an operation planeperpendicular to a reference plane determined by an origin of a thirddirection in the predetermined space pointed by the user and the firstdirection, a direction on an operation line through which anintersection where the third direction and the first direction intersectand the origin of the third direction pass.
 9. The informationprocessing device according to claim 1, wherein the control unitcontrols the display device to move the virtual object that is anoperation target in the depth direction on the line in the firstdirection and display the virtual object while maintaining the distancebetween the intersection where the first direction and the seconddirection intersect and the virtual object, and controls the displaydevice to display an image near the operation target in another window,on a basis of the determination result of the determination unit.
 10. Aninformation processing method including, by an information processingdevice: acquiring an operation angle that is an angle formed by a firstdirection in a predetermined space pointed by a user and a seconddirection in the predetermined space pointed by the user; setting, as areference angle, the operation angle acquired at a time point when aninstruction to start moving a virtual object on a line extending in thefirst direction is detected; determining whether or not the operationangle acquired in response to a change in the second direction is equalto or more than the reference angle; and controlling a display device tomove the virtual object in a depth direction on the line in the firstdirection and display the virtual object while maintaining a distancebetween an intersection where the first direction and the seconddirection intersect and the virtual object on a basis of a determinationresult.
 11. An information processing program causing a computer toexecute processing including: acquiring an operation angle that is anangle formed by a first direction in a predetermined space pointed by auser and a second direction in the predetermined space pointed by theuser; setting, as a reference angle, the operation angle acquired at atime point when an instruction to start moving a virtual object on aline extending in the first direction is detected; determining whetheror not the operation angle acquired in response to a change in thesecond direction is equal to or more than the reference angle; andcontrolling a display device to move the virtual object in a depthdirection on the line in the first direction and display the virtualobject while maintaining a distance between an intersection where thefirst direction and the second direction intersect and the virtualobject on a basis of a determination result.